A Brief Introduction to Duality Theory

These notes give an introduction to duality theory in the context of linear and positive semidefinite programming. These notes are based on material from Convex Analysis and Nonlinear Optimization by Borwein and Lewis and Numerical Optimization by Nocedal and Wright. Two examples are given to show how duality can be used. The first optimization application is to find the matrix in an affine family that has the least 2-norm. This problem can be formulated as the dual to a positive semi-definite program. The second problem is to find for an empirical correlation matrix Q̃ the closest positive semidefinite matrix Q with respect to the Frobenius norm. This problem can be formulated as a positive semidefinite program. We also derive it’s dual problem. Lastly, we briefly discuss some numerical methods useful in solving these constrained optimization problems. Cones An important construct that arises in optimization problems is a cone K. Cones are often used to formulate constraints in optimization problems and have some important properties with respect to the dual. We review here some basic definitions and results concerning cones. Definition 1 A cone K is a set invariant under multiplication by non-negative scalars. If x ∈ K and λ ≥ 0 then λx ∈ K. Definition 2 The dual of a cone is Dual(K) = {z ∈ R|zx ≥ 0} Three important cones are: Definition 3 Quadratic Cone: Kq = {z ∈ R| ‖(z2, . . . , zm)‖2 ≤ z1} Definition 4 Positive Orthant Cone: K+ = {z ∈ R| z1 ≥ 0, z2 ≥ 0, . . . , zm ≥ 0}